Preventing Unlearning During On-Line Training of Feedforward Networks1

Interference in neural networks occurs when learning in one area of the input space causes unlearning in another area. These interference problems are especially prevalent in on-line applications where learning is directed by training data that is currently available rather than some optimal presentation schedule of the training data. We propose a procedure that enhances a learning algorithm by giving it the ability to make the network more local and hence, less likely to suuer from future interference. Through simulations using Radial Basis Function (RBF) networks and sigmoidal, multi-layer perceptron (MLP) networks it is shown that by optimizing a new cost function that penalizes non-locality, the approximation error is reduced more quickly than with standard back-propagation.